MEMORANDUM April 17, 2015 
TO: Board Members 


FROM: Terry B. Grier, Ed.D. 
Superintendent of Schools 


SUBJECT: EFFECTIVE TEACHER FELLOWSHIP EVALUATION REPORT 


CONTACT: Carla Stevens, (713) 556-6700 


Attached is the 2014-2015 evaluation report on the Effective Teacher Fellowship (ETF) 
alternative certification program (ACP). This program evaluation compared the STAAR and 
Stanford 10 test performance of student samples whose teachers participated in four types of 
ACPs during the 2012-2013 and 2013-2014 academic years (ETF, university/community college, 
education service center, and private entity). The analyses of reading and mathematics 
achievement revealed mixed results, depending on the test and student grade level. However, 
among the most notable findings, 

e The 2012-2013 ETF ACP cohort outperformed students of teachers in comparison ACP 
cohorts as well as all HISD students on the 2013 STAAR 3-8 math tests (Level Il phase- 
in Il, percent met Satisfactory standards), while the university/community college ACP 
cohort and the Education Service Center cohort had the highest percentage of students 
who met Satisfactory performance standards on the 2013 STAAR 3-8 reading tests. 

e In addition, the 2013-2014 university/community college ACP cohort had the highest 
percentage of students who met Satisfactory standards on the 2014 STAAR 3-8 reading 
and math tests. 

e STAAR EOC reading, writing, and math exam results did not definitively identify a specific 
ACP type whose students consistently outperformed other ACP cohorts during the 2012- 
2013 and 2013-2014 academic years. 

e There was evidence that gaps are being reduced between students of ETF and non-ETF 
teachers based on value-added data reflecting student growth. 


Administrative Response: The immediate charge includes matching teacher content background 
to the certification area being pursued. This ensures that content expertise aligns to the teaching 
assignment, which is especially critical for those in secondary classrooms where the depth of 
knowledge and rigor exponentially increase. For early childhood through grade 6 and grades 4—8 
generalists, the charge is to ensure that ETF is working closely with school leaders to ensure that 
the best content match happens before a teacher is assigned a class/course. ETF will continue 
to use and modify its own behavioral selection tool as a predictor for teacher success and means 
to assess non-academic behaviors. Year-long professional learning will be re-sequenced so that 
it better meets the needs of diverse learners on day one of school. Instead of focusing on the more 
rigorous teaching skills later in the year, the data indicate that we need to focus on this while 
simultaneously working to train and support teachers in the foundational areas of classroom 
management and culture and backwards planning. Special emphasis is being placed on grades 
3—5 because of the correlation observed between classroom management and culture and student 
achievement. ETF is working to redesign the professional learning and support to these particular 


teachers via the ETF Management and Culture Teacher Development Specialist. Places that are 
doing ACP well will be studied. According to the National Council on Teacher Quality, some of 
these include: Los Angeles Unified School District Intern Program and Teach for America Boston. 


Should you have any questions or require any further information, please contact me or Carla 
Stevens in the Department of Research and Accountability, at 713-556-6700. 
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EFFECTIVE TEACHER FELLOWSHIP 
COMPARATIVE ANALYSIS OF ALTERNATIVE CERTIFICATION PROGRAMS AND STUDENT 
READING AND MATHEMATICS ACHIEVEMENT 
2014-2015 


Executive Summary 


The Effective Teacher Fellowship (ETF) is the Houston Independent School District's (HISD) 
Alternative Certification Program (ACP). ETF focuses on developing teachers in critical shortage areas, 
including reading, mathematics, bilingual education, and special education. ETF provides resources, such 
as online training for the Texas Examinations of Educator Standards™ (TExES™) and _ professional 
learning courses based on HISD’s current Instructional Practice and Professional Expectations Rubric 
(Effective Teacher Fellowship, n.d.). Instructional coaches deliver one-on-one support throughout the 
internship of the first-year teacher. Grant opportunities help to offset a percentage of the program cost for 
some participants. Applicants must complete all program requirements as determined by the Effective 
Teacher Fellowship, including a satisfactory year of teaching in an HISD classroom. 

ACPs are, traditionally, operated in universities, school districts, education service centers, 
community colleges, and private entities across Texas and throughout the United States. ACPs employ 
various strategies relative to recruitment, preparation, and school placement. For example, ETF adheres 
to the “Grow-Your-Own” model, which guides who is recruited into the program and the skill level of the 
applicant at entry. Another component of ETF is that teachers are typically assigned to schools with 
predominately at-risk students, which may not be a standard practice for other ACPs (Effective Teacher 
Fellowship, n.d.). 

As a nontraditional route to teacher certification, understanding the extent that various ACPs, 
particularly ETF, contribute toward student learning in HISD is important for planning, decision making, 
and assessing whether the academic performance needs of students taught by ACP teachers are 
adequately and effectively met. To that end, this evaluation addressed the following research questions. 

e Did students of the 2012—2013 and the 2013-2014 ETF ACP cohorts attain higher reading and 
mathematics achievement than students of teachers who participated in university/community 
college, education service center, and private entity ACPs? 

e What was the impact of teacher ACPs on student academic progress based on EVAAS value- 
added results? 

e What factors were the strongest predictors of students’ reading and mathematics performance, 
considering teachers’ ACP status and students’ demographic characteristics among traditionally 
underperforming student groups? 


The study samples were students identified as having a teacher in only one of the four ACPs’. This 
was done to control for confounding effects and to differentiate how much of the observed effect on 
outcomes was due to student’s exposure to a teacher who participated in a specific ACP rather than a 
combination of ACPs (What Works Clearinghouse, Procedures and Standards Handbook, version 2.1, 
2010). Student academic performance outcomes included 2013 and 2014 English STAAR, STAAR End- 
of-Course, Stanford 10, and EVAAS value-added gain index scores for students of the respective 2012-— 
2013 and the 2013-2014 ACP teacher cohorts. STAAR data were based on Level II phase-in | 
Satisfactory performance standards. 


i See Appendix A for specific ACPs of teachers in this evaluation. 
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Highlights 


During the 2012-2013 academic year, 699 staff were employed in HISD as participants of ACPs. 
A slight increase was noted during the 2013-2014 academic year to 713 staff. These figures 
include individuals acquiring teacher and counselor ACPs, for example. Student test performance 
for 547 teachers in the 2012-2013 ACP cohorts, and 628 teachers in the 2013-2014 ACP 
cohorts were analyzed in this report. 


An overwhelming majority of the student samples were students of Private entity ACP teachers in 
the 2012-2013 and the 2013-2014 ACP teacher cohorts. However, students of ETF cohorts were 
more likely to be economically disadvantaged and less likely to be classified as Special Education 
than students in comparison ACP cohorts. 


Among the 2012-2013 ACP cohorts, the highest percentage of students who met Level II phase- 
in 1 Satisfactory standards on the 2013 STAAR reading test were students of 
university/community college and education service center ACP teachers; slightly outperforming 
the ETF cohort by two percentage points (69 percent and 67 percent, respectively). Grades three 
through eight analyses on the test revealed that the ETF student group attained higher mean 
reading scale scores than the non-ETF student group at sixth and seventh grades. The difference 
was Statistically significant at seventh grade (p < .05). 


On the 2013 STAAR English math test, students of the 2012-2013 ETF ACP cohort 
outperformed students of teachers in comparison ACP cohorts as well as all HISD students at 68 
percent met Satisfactory standards (Level II phase-in 1). 


On the 2014 STAAR English reading and math tests, students of the 2013-2014 
university/community college ACP cohort achieved the highest percent met Satisfactory at the 
Level Il phase-in 1 standard compared to students of comparison ACP cohort teachers (72 
percent on both the reading and math tests). This ACP cohort outperformed students of the ETF 
cohort by 11 and 8 percentage points in reading and math, respectively. 


The EOC exam results for the 2012-2013 ACP cohorts revealed that the university/community 
college cohort had the highest percentage of students who met Satisfactory (Level Il phase-in 1 
standard) on the 2013 ELA | - Reading, ELA | - Writing, and Algebra | EOC exams. The 
difference between the ETF ACP cohort and the university/community college cohort was 25, 35, 
and 7 percentage points, respectively. However, students of ETF ACP teachers attained a higher 
mean scale score than the non-ETF student group on the Algebra | EOC exam. This difference 
was Statistically significant (p < .05). 


On the 2014 English | EOC exam, students of the 2013-2014 Private entity ACP cohort achieved 
the highest percent met Satisfactory standard (Level II phase-in 1), outperforming students of 
ETF ACP teachers by eight percentage points (52 percent vs. 44 percent). Scale score analysis 
revealed that students of ETF teachers attained a lower mean scale score compared to students 
of non-ETF teachers on the English | EOC exam. The difference between ETF and non-ETF 
cohort students’ mean English | EOC exam scale scores was statistically significant (p < .05). 


On the 2014 Algebra | EOC exam, students of the 2013-2014 ESC ACP cohort attained the 
highest percent met Satisfactory standard (Level Il phase-in 1), outperforming students of the 
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ETF cohort by eight percentage points (78 percent vs. 70 percent). At the same time, students of 
ETF teachers attained a lower mean Algebra | scale score than students of non-ETF teachers on 
the exam. The difference in the mean Algebra | EOC scale score of ETF and non-ETF cohort 
students was statistically significant (p < .05). 


e On the 2013 Stanford 10 test, the mean NCE for students of the 2012-2013 ETF ACP cohort 
compared to other ACP cohorts was the lowest in reading (41 NCEs), but the highest among the 
ACPs in math (52 NCEs). 


e On the 2014 Stanford 10 reading test, students of the ETF cohort had the lowest mean NCE 
among the ACP cohorts (38 NCEs), which is slightly below average. On the 2014 Stanford 10 
math test, students in the 2013-2014 university/community college ACP cohort attained a higher 
mean math NCE score than other cohorts, four NCEs higher than the ETF cohort (51 NCEs vs. 
47 NCEs). 


e Value-added analyses based on teachers’ composite gain indexes revealed that, for the 2012— 
2013 and the 2013-2014 cohorts, a higher percentage of university/community college ACP 
teachers attained progress scores that were within the “well above” average range because their 
students made substantially more progress than the standard for academic growth (13 and 16 
percent, respectively) compared to other ACP cohorts. 


e Multiple regression analysis using 2013 STAAR reading and math scale scores for grades 3 
through 8 revealed that the 2012-2013 ETF student group consistently had higher reading and 
math scale scores than the non-ETF student group (Beta = .021 for reading and Beta = .072 for 
math). In addition, traditionally underperforming student groups (i.e., Black vs. non-Black, 
Hispanic vs. non-Hispanic, economically disadvantaged, Special Education, and at risk students) 
consistently had lower reading and math scale scores than their counterparts. This finding was 
also evident for LEP students in reading. 


e Multiple regression analysis based on 2014 STAAR grades 3 through 8 results revealed that the 
ETF student group consistently had lower reading and math scale scores than the non-ETF 
student group (Beta = -.107 for reading and Beta = -.081 for math). Beta coefficients indicated 
that traditionally underperforming student groups (i.e., Black vs. non-Black, Hispanic vs. non- 
Hispanic, economically disadvantaged, Special Education, and at risk students) consistently had 
lower reading and math scale scores than their counterparts. In addition, LEP students had 
consistently lower reading scores but higher math scores than non-LEP students in the model 
relative to the constant. 


Recommendations 


1. Mixed reading and math performance was evident for students in the study samples across tests and 
grade levels, regardless of ACP cohort. However, results were consistent with prior research, in that, 
traditionally underperforming students, regardless of ACP (Black, Hispanic, economically 
disadvantaged, at risk, Special Education, and LEP), had lower STAAR reading and math scores 
than their counterparts. ETF program administrators should consider focusing professional 
development on best practices that address students’ diverse learning styles and culture by 
incorporating evidence-based models (e.g., Universal Design for Learning) that are designed to 
improve performance for all students. Additional considerations should be made regarding teachers’ 
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skills and background knowledge in the selection and placement processes that may effectively 
contribute toward learning for high-need students. 

2. Although students of different ACP cohorts were assessed in this evaluation, there was evidence that 
gaps are being reduced between students of ETF and non-ETF teachers based on value-added data 
reflecting student growth. Moreover, students of ETF teachers achieved higher scale scores at some 
grade levels in reading and math compared to non ETF-teachers. Program administrators should 
consider examining program strategies that build content knowledge and pedagogy of teachers at 
each grade level to strengthen alignment with TEKS. 


Administrative Response 

Research findings contend that ETF teachers teach higher numbers of economically disadvantaged 
and English Language Learner (ELL) students, and this aligns with our placement model. As a “grow 
your own” district talent pipeline, it is critical that we do all we can to stand in the gap for students who 
have historically underperformed, and the research shows that some gaps are in fact being reduced by 
ETF teachers. We have a charge to ensure that ETF ACP exceeds other mechanisms, and there’s a 
mixed level of evidence—depending on grade level and content. The immediate charge includes the 
following commitments: 


e Match teacher content background—undergraduate or graduate—to the certification area being 
pursued. This ensures that content expertise aligns to the teaching assignment, and this is 
especially critical for those in secondary classrooms where the depth of knowledge and rigor 
exponentially increase. For early childhood through grade 6 and grades 4—8 generalists, the 
charge is to ensure that ETF is working closely with school leaders to ensure that the best 
content match happens before a teacher is assigned a class/course in the master schedule. ETF 
will continue to use and modify its own behavioral selection tool as a predictor for teacher 
success and means to assess non-academic behaviors. 


e Re-sequence year-long professional learning so that it better meets the needs of diverse learners 
on day one of school. Instead of focusing on the more rigorous teaching skills later in the year, 
the data tells us that we need to focus on this while simultaneously working to train and support 
teachers in the foundational areas of classroom management and culture and backwards 
planning. 


e Special emphasis is being placed on grades 3-5 because of the correlation observed between 
classroom management and culture and student achievement. ETF is working to redesign the 
professional learning and support provided these particular teachers via the ETF Management 
and Culture Teacher Development Specialist. 


e Places that are doing ACP well will be studied. According to the National Council on 


Teacher Quality, some of these include: Los Angeles Unified School District Intern Program and 
Teach for America Boston. 
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Introduction 


Alternative certification programs (ACPs) offer a nontraditional route to teacher certification for 
individuals who are interested in becoming a teacher. ACPs are located in various organizations, 
including universities, school districts, education service centers, community colleges, and private entities. 
The HISD ETF ACP was implemented in 1985 in response to the 1984 Texas Legislature’s House Bill 72 
(Tex. H.B. 72, 68th Leg. 2d C.S.) (Effective Teacher Fellowship, n.d.). An intent of the bill was to enhance 
equity and effectiveness in the state’s education system by improving teacher quality (Grubbs, 1985). The 
bill addressed teacher recruitment, retention, and education, along with the development of ACPs that 
varied from the traditional university teacher certification program. 

Over a one-year program implementation period, the ETF ACP focuses on developing teachers in 
critical shortage areas, including bilingual education, special education, and secondary math and science 
(Effective Teacher Initiative, n.d.). Resources are provided, such as online training for the Texas 
Examinations of Educator Standards™ (TExES™) content exams and professional learning courses 
based on HISD’s current Instructional Practice and Professional Expectations Rubric. Instructional 
coaches provide one-on-one support throughout the internship of the first-year teacher. Grant 
opportunities help to offset a percentage of the program cost for some participants. 

Annually, HISD hires teachers who participate in ACP programs that operate across the United 
States, including the ETF ACP. During the 2012-2013 academic year, 699 teachers acquired 
certifications through ETF, with a slight increase to 713 teachers during the 2013-2014 academic year. 
Expenses incurred by the ETF ACP are depicted in Table 1 for the 2012-2013 and the 2013-2014 
academic years. An increase in teacher participants is accompanied by an increase in overall program 
revenue and operating costs. 


Table 1. Expenditures by Academic year of the HISD ETF ACP Program, 2012-2013 and 2013-2014 


PNor-(e(-Vinl om Actsls 7) 


Expense Item 


2012-2013 2013-2014 Increase 
Payroll $687,585.44 $983,696.99 43.1% 
Contracted Services $28,025.67 $33,436.35 19.3% 
Supplies Materials & Food $27,461.57 $33,108.32 20.6% 
Travel & Fees $27,620.09 $43,557.19 57.7% 
Technology $1,977.20 $10,258.00 418.8% 
Total $772,669.97 | $1,104,056.85 42.9% 
Number of ACP participants (includes counselors and 699 713 2.0% 
educational diagnosticians) 


While all of the ACPs in Texas are required by state law to offer intensive, focused classroom 
training, the length of time that teachers are trained varies (Education Week, 2004). During the minimum 
one-year classroom instructional time requirement, ACP participants can teach as paid interns with 
supervision and mentoring. Some programs offer a non-paid clinical experience similar to student 
teaching in lieu of a paid internship. The ETF ACP has specific eligibility guidelines which are described in 
Appendix B. 


HISD Research and Accountability 5 


Regardless of the certification route, teachers are expected to positively impact student achievement. 
While there have been studies that investigated the context of instruction in teacher education programs 
and what they learn from these opportunities (Abell, Arbaugh, Chval, Friedricshen, Lannin, Volkmann, 
n.d.), little research has been done on how teachers certified through ACPs impact student learning. 
Thus, research on which ACPs produce effective teachers and whose students perform well or better 
than their peers is important for planning quality ACP programs. To that end, the purpose of this 
evaluation was to assess the impact of the ETF ACP on student achievement compared to the 
performance of students whose teachers participated in university/community college, ESC, or private 
entity ACPs. 


Review of the Literature 


Numerous studies have shown that “highly qualified” classroom teachers are pivotal in fostering 
student achievement (Ingersoll, 2001; Darling-Hammond, 2002). The No Child Left Behind Act stipulates 
that in order for teachers certified through alternative routes to be considered “highly qualified,” the routes 
must consist of sustained, intensive, and classroom-focused professional development before and while 
teaching (Education Week, 2004). In addition, ACPs must provide structured support, such as a teacher 
mentoring program. The influence of high-quality mentoring and support on teacher development has 
been documented in the research (Darling-Hammond, 1990; Zientek, Reichwein; Gamba, & Capraro, 
2005). 

An assumption is that teacher effectiveness may be partly a function of specialized training in how to 
teach, which may be acquired through ACPs (Ballou & Podgursky, 2000; U.S. Department of Education, 
2002). However, some ACP models allow participants to earn their teaching certificates in a shorter time 
frame than teachers who enrolled in a four-year undergraduate teacher education program (Education 
Week, 2004). These ACP teachers may be allowed to teach while completing the program requirements 
for such alternative certification. 

Supporters of ACPs maintain that alternative routes of entering the teaching profession are an 
effective strategy for recruiting teachers and managing teacher shortages (Education Week, 2004; Shen, 
1999, Darling-Hammond, 2002; Laczko-Kerr and Berliner, 2002). While alternative routes of teacher 
certification may play a critical role in expanding the pool of teachers, the research varies regarding 
whether ACPs make a significant difference in student performance (Allen, 2003; Boyd, Grossman, 
Lankford, & Wyckoff, 2008). Alternative teacher certification routes that are well-designed can enhance 
workforce diversity and attract teacher candidates who are experts in subject-matter that students need to 
be academically successful in school (Hess, 2001; Roach & Cohen, 2002). However, teachers who are 
not adequately-prepared to provide instruction can hinder student learning (Darling-Hammond, 2002; 
Laczko-Kerr and Berliner, 2002). A contrasting perspective is that ACPs “shortchange both the teacher 
candidates and students they teach because their preparation, particularly pedagogy, is inadequate” 
(Allen, 2003, p. 3). 


Methods 


Data Collection 
e Teacher and student linkages for the 2012-2013 and the 2013-2014 ACP cohorts were extracted 


from the Chancery database based on the year of the cohort. Test data used in this report were 
computed for students identified through teacher-student linkages to form the study samples. To 
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control for confounding effects and to differentiate how much of the observed effect of outcomes 
measured were due to student’s exposure to a specific ACP program rather a combination of 
ACP programs, the results of students identified as having a teacher in only one of the four ACP 
programs were analyzed in this evaluation (i.e., ACP group assignment was mutually exclusive) 
(What Works Clearinghouse, Procedures and Standards Handbook, version 2.1, 2010). 

For the 2012—2013 ACP cohorts, student data for 547 teachers were included in the analysis 
(125 teachers were represented in ETF, 35 teachers in university/community college, 65 teachers 
in ESC, and 322 teachers in the private ACP cohorts). For the 2013-2014 ACP cohorts, student 
data for 628 teachers were included in the analysis (141 teachers in ETF, 46 teachers in 
university/community college, 49 teachers in ESC, and 391 teachers in the private ACP cohorts). 
STAAR is a state-mandated, criterion-referenced assessment used to measure student 
achievement. The Level II phase-in | Satisfactory standard on STAAR reflects whether students 
are sufficiently prepared for the next course. Descriptive statistics depicted in this report included 
frequencies and percentages of students who met Level II phase-in | Satisfactory standards or 
scored at the Advanced level on the regular English language version of the STAAR and End-of- 
Course (EOC) assessments in reading and mathematics during the spring, first test 
administration. Mean STAAR scale scores were also presented by grade level. (Spanish version 
of STAAR was not analyzed due to low numbers of students tested in some ACP teacher/student 
cohorts, limiting the extent that reliable comparisons could be made with the ETF cohort.) 
Stanford 10 Normal Curve Equivalents were analyzed for students with both reading and math 
scores. There were 22,130 students in the 2012—2013 ACP teacher/student cohort and 22,170 
students in the 2013-2014 ACP teacher/student cohorts who had reading and/or math scores. 
Data were presented by grade level, mean NCEs, and standard deviations for each ACP group. 
Independent samples t-test analysis was conducted using STAAR scale scores and Stanford 10 
NCEs to determine whether there were statistically significant differences between the ETF and 
non-ETF groups at the p < .05 level using IBM SPSS software. STAAR scale scores allow direct 
comparisons of student performance between specific sets of test questions from different test 
administrations (Texas Education Agency, 2014). 

Value-added analysis used in this report was calculated using the EVAAS® model. Value-added 
analysis compares the change in achievement of a group of students from one year to the next 
with an expected amount of change based on the students’ prior achievement history. HISD uses 
value-added analysis to measure the impact of the district's curriculum and instruction on 
students’ academic progress from year to year. Teacher’s composite gain index was used in the 
analysis. The gain index is calculated by dividing the growth measure by the standard error (HISD 
Department of Research and Accountability website). Interpretation of the composite gain index 
can be found in Appendix J. 

Linear mixed-effects modeling was used to determine predictors of students’ STAAR reading and 
math scores. Preliminary analyses were performed to ensure no violation of the assumptions of 
normality, linearity, multicollinearity, and homoscedasticity. The model controlled for economic 
status, at risk status, race, gender, and whether students were classified as special education. 


Limitations 
There are several limitations to the study. The limitations are as follows. 


Results of students identified as having a teacher in only one of the four ACP programs were 
analyzed in this evaluation. This was done to control for confounding effects and to differentiate 
how much of the observed effect on outcomes was due to student’s exposure to a teacher who 
participated in a specific ACP rather than a combination of ACPs (What Works Clearinghouse, 
Procedures and Standards Handbook, version 2.1, 2010). 
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e Analyses did not take into account students’ previous year’s performance and its impact on 
current performance. 

e As a result of the emphasis on teacher certification route, ACP teachers may have been 
concentrated in specific types of schools, which could have had a disproportional impact on 
student achievement. 

e ACPs may vary in how the program is implemented; therefore, grouped data may mask student 
learning effects of individual ACPs. 


Results 


What was the profile of ACP cohort students in the study sample, 2012-2013 and 2013-2014? 


Figure 1 presents the total number of students in the study sample by their teacher’s ACP. The 
demographic characteristics of these students can be found in Appendix C. A profile of the students by 
cohort year and teacher ACP is presented below. 

e The highest number of students in both the 2012—2013 and the 2013-2014 ACP cohorts were 
students of private entity ACP teachers. ETF had the second highest number of students in the 
study sample in both years. 

e For the 2012-2013 ACP cohorts, students of ETF, ESC, and Private entity teachers were 
predominately Hispanic. ETF teachers had the highest percentage of students who were 
economically disadvantaged (89.7 percent) and LEP (43.7 percent), but the lowest percentage 
of students classified as Special Education (6.2 percent). The ESC cohort had the highest 
percent of at-risk students (52.3 percent) compared to ETF and private entity cohorts (49.6 
percent) and the university/community college cohort (42.3 percent). 

e Similar to the previous year cohorts, Hispanic students were more prevalent among students of 
teachers in the 2013-2014 ACP cohorts. In addition, ETF had the highest percentage of 
economically-disadvantaged (87.4 percent) and at risk (69.9 percent) students; whereas, the 
private entity cohort had the highest percentage of students classified as Special Education (8.8 
percent) among the ACPs. 


Figure 1. Number of students in the study sample, 2012-2013 and 2013-2014 ACP cohorts 
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Did students of 2012-2013 and the 2013-2014 ETF ACP cohorts attain higher reading and 
mathematics achievement than students of teachers who participated in university/community 
college, education service center, and private entity ACPs? 


The academic performance of students whose teachers were ETF, university/community college, 
ESC, or private ACP teachers in 2012-2013 and 2013-2014 was measured using the regular English 
STAAR at the met Level Il, phase in 1 Satisfactory standards in reading and mathematics. STAAR scale 
score analysis was also conducted. Combined results for grades three through eight are presented here, 
while grade level passing rates and scale score means can be found in Appendix D. Additional analyses 
of ETF compared to non-ETF ACP cohorts are also presented in this report. 


STAAR English Reading and Math Performance, 2012-2013 ACP Cohorts 


STAAR English Reading Grades 3 - 8 

e Among the 2012-2013 ACP cohorts, Figure 2 shows that the highest percentage of students 
who met Satisfactory on the 2013 STAAR English reading test were students of 
university/community college and education service center ACP cohorts (69 percent). Students of 
the ETF ACP cohort attained the lowest percentage of students who met Satisfactory on the 
reading test (67 percent). 

e The percent of students who met Satisfactory on the 2013 reading STAAR was slightly higher for 
all HISD students compared to all ACP cohorts. The difference in performance of all HISD 
students and students of ETF teachers was three percentage points (Figure 2). 


Figure 2. Percent met Satisfactory, English STAAR reading and math, 
2012-2013 ACP cohorts, spring 2013 
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e STAAR English reading scale score analysis by grade level for students of ETF teachers 
compared to students of non-ETF teachers reveals that the ETF student group attained higher 
mean reading scale scores at the sixth and seventh grades. The differences in performance at 
the third through fifth, seventh, and eighth grades were statistically significant (p < .05) (Figure 3, 
Appendix D). 


Figure 3. 2013 English STAAR reading scale score analysis, 
2012-2013 ETF vs. Non-ETF ACP 


Mean Scale Score 


Note: Statistical significance on 2013 English STAAR reading at grades 3, 4, 5, 7, and 8 (p < .05). 


STAAR English Math Grades 3 - 8 

e On the STAAR English math test, students of the 2012-2013 ETF ACP cohort outperformed 
students of teachers in the comparison cohorts and all HISD students (Figure 2). The difference 
in the percent of ETF cohort students who met Satisfactory (68 percent) was six percentage 
points higher than students in the lowest performing cohort, two percentage points higher than 
students in the highest performing cohort, and one percentage point higher than all HISD 
students. 

e STAAR math scale score analysis by grade level (Figure 4) reveals that students of the ETF 
cohort attained higher mean scale scores than students of the non-ETF cohort at grades five 
through eight. These differences were statistically significant at grade three as well as at grades 
five through eight (p < .05) (Appendix D). 
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Figure 4. 2013 English STAAR math scale score analysis, 
2012-2013 ETF vs. Non-ETF 
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Note: Statistical significance on 2013 English STAAR math at grades 3, 5, 6, 7, and 8 (p < .05). 


STAAR Reading and Math Performance, 2013-2014 ACP Cohorts 


STAAR English Reading Grades 3 - 8 
e Figure 5 shows that students of the 2013-2014 university/community college ACP cohort 
achieved the highest percent met Satisfactory standard on the 2014 STAAR English reading test, 
outperforming students of ETF ACP teachers by 11 percentage points (72 percent vs. 61 
percent). 
e All HISD students outperformed students of the 2013-2014 ETF cohort by eight percentage 
points on the 2014 STAAR reading test (69 percent vs. 61 percent). 


Figure 5. 2014 English STAAR reading and math results, 2013-2014 ACP cohorts 
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STAAR reading scale score analysis by grade level for the 2013—2014 ACP cohorts can be found 
in Figure 6. It is evident that students of the ETF cohort attained a higher mean reading scale 
score than the non-ETF comparison group at fourth grade only. Statistically significant differences 
in the performance in the ETF student group relative to the non-ETF student group were detected 
at third through sixth grades (p < .05) (Appendix E). 


Figure 6. 2014 English STAAR reading scale score analysis, 
2013-2014 ETF vs. Non-ETF ACP 
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Note: Statistical significance on 2014 English STAAR reading at grades 3, 4, 5, and 6 (p < .05). 


STAAR English Math Grades 3 - 8 


On the 2014 STAAR English math test (Figure 5), students of the 2013-2014 
university/community college cohort attained the highest percent met Satisfactory standard (72 
percent); whereas, the ETF cohort had the lowest percent of students who met the standard (64 
percent). 

The all HISD student group outperformed the ETF group by five percentage points in 2014 (69 
percent vs. 64 percent). 

2014 English STAAR math scale score analysis by grade level (Figure 7) shows that the ETF 
student group attained a higher mean math scale score compared to the non-ETF student group 
at fourth grade. The difference in the mean math score at fourth grade was statistically significant 
(p < .05). Statistical significance was also noted at third, fifth, and sixth grades in favor of the non- 
ETF student group (Appendix E). 
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Figure 7. 2014 English STAAR math scale score analysis, 
2013-2014 ETF vs. Non-ETF ACP 
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Note: Statistical significance on 2014 English STAAR math at grades 3, 4, 5, and 6 (p < .05). 


STAAR End-of-Course (EOC) Exams 


STAAR EOC exam analysis included English language arts (ELA) and math test results for seventh 
through tenth-grade students at the Level II phase-in 1 standard. The results of students whose teachers 
were 2012-2013 and 2013-2014 ACP teachers are presented by program. 


STAAR EOC, 2012-2013 ACP Cohort 
Figure 8 depicts student performance by 2012-2013 cohorts on the ELA | - Reading, ELA | - Writing, 
and Algebra | EOC exams and for all students tested in the district. Additional data can be found in 
Appendix F. 
e Students of teachers in the 2012-2013 university/community college cohort attained the highest 
2013 ELA | - Reading, ELA | - Writing, and Algebra | EOC exam results among the ACP cohorts. 
This group’s performance exceeded the ETF group by 25 percentage points on the ELA | - 
Reading exam, 35 percentage points on the ELA | - Writing exam, and 7 percentage points on the 
Algebra | exam (Figure 8). 
e STAAR EOC scale score analysis by exam (Figure 9) shows that students of the ETF cohort 
attained a higher mean Algebra | scale score than students of non-ETF teachers. The difference 
in the mean Algebra | scale scores was statistically significant (p < .05) (Appendix F). 
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Figure 8. STAAR EOC ELA | - Reading, ELA | - Writing, and math exams, 2012-2013 ACP cohorts, spring 
2013 
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Figure 9. STAAR EOC Algebra I, ELA I - Reading, and ELA | - Writing scale score analysis, 
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Note: Statistical significance on 2013 STAAR Algebra | EOC exam (p < .05). 


STAAR EOC, 2013-2014 ACP Cohort 


Figure 10 depicts student performance for 2013-2014 ACP cohort teachers on the newly-revised 
English | and Algebra | EOC exams. Additional data can be found in Appendix G. 
e Among the 2013-2014 cohorts, the highest percentage of students who met Satisfactory on the 
2014 English | exam were students of private entity ACP teachers compared to students in other 

ACP cohorts. The percentage of ETF cohort students who met Satisfactory on the exam was 

eight percentage points lower than Private entity ACP cohort students based on these standards 

(52 percent vs. 44 percent) (Figure 10). 
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e On the Algebra | exam, the percentage of students who met Satisfactory was highest for students 
of the 2013-2014 ESC ACP cohort compared to other cohort students. The percentage of ETF 
cohort students who met Satisfactory on the test was eight percentage points lower than the ESC 
cohort (78 percent vs. 70 percent) (Figure 10). 

e STAAR EOC scale score analysis by exam (Figure 11) shows that students of ETF teachers 
attained lower mean Algebra | and English | scale scores than students of non-ETF teachers. The 
differences in mean Algebra | and English EOC scale scores were statistically significant in favor 
of non-ETF students (p < .05) (Appendix G). 


Figure 10. STAAR EOC English | and Algebra | results, 
2013-2014 ACP cohort, spring 2014 
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Figure 11. STAAR EOC Algebra I and English | scale score analysis, 
2013-2014 ETF vs. Non-ETF, spring 2014 
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Note: Statistical significance on the 2014 STAAR Algebra I and English | EOC exams (p < .05). 


HISD Research and Accountability 15 


Stanford 10 Achievement Test, 2012-2013 and 2013-2014 ACP Cohorts 


Stanford 10 results for 2012-2013 ACP cohorts in reading and math are provided in Figure 12. More 
detail analyses is presented in Appendix H. 

e The mean reading normal curve equivalents (NCEs) ranged from 41 to 42 NCEs and from 50 to 
52 NCEs in math for students of ACP cohorts. These scores were within the average range 
(between 40 — 60 NCEs). 

e Although slight, the mean NCE for students of the 2012-2013 ETF ACP cohort compared to other 
ACP cohorts was the lowest in reading (41 NCEs), but the highest in math (52 NCEs). 

e The mean NCE for HISD was higher than students of the ETF ACP cohort (45 NCEs vs. 41 
NCEs), but comparable in math (52 NCEs for both student groups). 


Figure 12. Mean 2013 Stanford 10 Reading and Math NCEs, 2012-2013 ACP Cohorts 
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Stanford 10 results for students of the 2013-2014 ACP cohorts in reading and math are provided in 
Figure 13. More detail analyses is shown in Appendix I. 
e Students in the 2013-2014 university/community college ACP cohort attained the highest mean 


reading NCE score compared to other ACP cohorts. Students in the ETF cohort attained the 
lowest mean NCE score in reading (38 NCEs), which is slightly below average performance. 


Figure 13. Mean 2014 Stanford 10 Reading and Math NCEs, 2013-2014 ACP Cohorts 
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e On the Stanford 10 math test, students in the 2013-2014 university/community college ACP 
cohort attained a slightly higher mean math NCE score than the other ACP cohorts, which was 
four NCEs higher than the ETF cohort (51 NCEs vs. 47 NCEs). HISD student performance was 
comparable to the university/community college ACP cohort. 


What was the impact of teacher ACPs on student academic progress based on EVAAS value- 
added results? 


Value-added, 2012-2013 and 2013-2014 ACP Cohorts 

Value-added analysis was conducted to compare the change in performance of student groups by 
ACP teacher from one year to the next. Teachers’ composite gain indices were used in the analysis. The 
gain index is calculated by dividing the growth measure by the standard error. An explanation of the 
measure is depicted in Appendix J. 

e Figure 14 shows that, for the 2013-2014 ACP cohorts, a higher percentage of 
university/community college ACP teachers attained gain index scores that were within the “well 
above” average range (16.0 percent). Students in this group made substantially more progress 
than the Standard for Academic Growth by 2 standard errors or more. Comparatively, 6.0 percent 
of the ETF ACP cohort received student progress scores that fell within the “well above” average 


range. 
Figure 14. Composite Gain Index Score, 2012-2013 ACP Cohorts 
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e Figure 15 shows that, for the 2013-2014 cohorts, a higher percentage of university/community 
college ACP teachers attained gain index scores that were within the “well above” average range 
(13.0 percent). Comparatively, 8.9 percent of the ETF ACP cohort received gain index scores that 
fell within the “well above” average range. 
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e A comparison of composite gain index results for the 2012-2013 and the 2013-2014 ACP 
cohorts revealed a reduction in the gap between the percentage of ETF and university/community 
college ACP teachers who scored “well above” average (6.0 and 16.0 percent, respectively 
compared to 8.9 and 13.0 percent, respectively). 


Figure 15. Composite Gain Index Score, 2013-2014 ACP Cohorts 
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What factors were the strongest predictors of students’ reading and mathematics achievement, 
considering teachers’ ACP status and students’ demographic characteristics among traditionally 
underperforming student groups? 


Tables 2 and 3 present multiple regression analyses using English STAAR reading and math scale 
scores for grades 3-8 as the dependent variables and ETF vs. non-ETF, gender, economic status, at-risk 
status, special education classification and LEP status as independent variables. The ethnicity of 
traditionally lower performing students (Black vs. non-Black and Hispanic vs. non-Hispanic) were also 
included in the model as independent variables. Beta coefficients and p-values are depicted in Tables 2 
and 3. 


Table 2. Multiple Regression Analysis Predicting 2013 STAAR Reading and Math Scale Score Performance 


STAAR English Reading STAAR English Math 
Total N = 18,907 Total N = 17,683 
Beta p B p 
ACP (ETF = 1, non-ETF = 0) 021 .002 .072 .000 
Gender -.038 .000 .022 .002 
Economic Status -.159 .000 -.122 .000 
At Risk -.100 .000 -.195 .000 
Special Education -.107 .000 -.111 .000 
LEP -.268 .000 .004 .655 
Black -.304 .000 -.366 .000 
Hispanic -.160 .000 -.221 .000 
Adjusted R* = .219 Adjusted R” = .153 
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Multiple Regression Analysis 


For the 2012—2013 ACP cohorts, all of the independent variables in the model made a significant 
unique contribution toward the prediction of students’ 2013 STAAR reading scale scores; and all 
of the independent variables in the model, except LEP status, made a significant unique 
contribution toward the prediction of students’ 2013 STAAR math scale scores. 

The ETF student group consistently had higher reading and math scale scores than the non-ETF 
student group (Beta = .021 for reading and Beta = .072 for math). 

Beta coefficients indicated that traditionally underperforming student groups (i.e., Black vs. non- 
Black, Hispanic vs. non-Hispanic, economically disadvantaged, Special Education, and at risk 
students) consistently had lower reading and math scale scores than their counterparts. This 
finding was also evident for LEP students in reading. 


Table 3 presents multiple regression analysis using 2014 STAAR reading and math scale scores for 
grades 3-8 as the dependent variable and student demographic characteristics as the independent 
variables. 


For the 2013-2014 ACP cohorts, all of the independent variables in the model made a significant 
unique contribution toward the prediction of students’ 2014 STAAR reading scale scores; and all 
of the independent variables in the model, except gender, made a significant unique contribution 
toward the prediction of students’ 2014 STAAR math scale scores. 

The ETF student group consistently had lower reading and math scale scores than the non-ETF 
student group (6 = -.107 for reading and B = .081 for math) relative to the constant. 

Beta coefficients indicated that traditionally underperforming student groups (i.e., Black vs. non- 
Black, Hispanic vs. non-Hispanic, economically disadvantaged, Special Education, and at risk 
students) consistently had lower reading and math scale scores than their counterparts. In 
addition, LEP students had consistently lower reading scores but higher math scores than non- 
LEP students in the model relative to the constant. 


Table 3. Multiple Regression Analysis Predicting 2014 STAAR Reading and Math Scale Score Performance 


STAAR English Reading STAAR English Math 
Total N = 18,843 Total N = 17,692 
Beta Beta p 
ACP (ETF = 1, non-ETF = 0) -.107 .000 -.081 .000 
Gender -.035 .000 .003 .626 
Economic Status -.127 .000 -.075 .000 
At Risk -.387 .000 -.389 .000 
Special Education -.054 .000 -.060 .000 
LEP -.094 .000 .046 .000 
Black -.284 .000 -.339 .000 
Hispanic -.132 .000 -.170 .000 
Adjusted R” = .292 Adjusted R” = .250 


Discussion 


The research indicates that alternative routes of teacher certification may play a critical role in 
expanding the pool of teachers (Education Week, 2004); however, the impact of ACPs on student 
performance is not definitive (Allen, 2003; Boyd, Grossman, Lankford, & Wyckoff, 2005). The Effective 
Teacher Fellowship Alternative Certification Program (ETF) was designed to address critical shortage 
areas in HISD that are specific to students’ academic abilities and special program needs (i.e., bilingual 
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education, and special education). This evaluation was designed to provide insight into the extent that 
ACPs contribute toward student learning in order to facilitate successful planning of a quality ACP 
program in HISD. 

This program evaluation compared the test performance of student samples whose teachers 
participated in four types of ACPs during the 2012-2013 and 2013-2014 academic years (ETF, 
university/community college, ESC, and private entity). The majority of students in the samples were 
students of private entity ACP teachers. At the same time, the ETF cohorts had the highest percentage of 
students who were economically-disadvantaged for both cohort years. 

The analyses of reading and mathematics achievement revealed mixed results, depending on the test 
and student grade level. However, among the most notable findings, the 2012-2013 ETF ACP cohort 
outperformed students of teachers in comparison ACP cohorts as well as all HISD students on the 2013 
STAAR math tests (Level Il phase-in 1, percent met Satisfactory standards), while the 
university/community college ACP cohort and ESC had the highest percentage of students who met 
Satisfactory performance standards on the 2013 STAAR reading tests. In addition, the 2013-2014 
university/community college ACP cohort had the highest percentage of students who met Satisfactory 
standards on the 2014 STAAR reading and math tests. The majority of grades 3 through 8 ETF student 
groups outperformed non-ETF students at comparable grade levels for the 2013-2014 cohorts. EOC 
reading, writing, and math exam results did not definitively identify a specific ACP type whose students 
consistently outperformed other ACP cohorts during the 2012—2013 and 2013-2014 academic years. The 
lack of control for students’ previous years’ academic performance limited the interpretation of these 
results. Nevertheless, considering ETF’s “Grow-Your-Own” model, there was evidence that gaps are 
being reduced between ETF and non-ETF teachers based on value-added data reflecting student growth. 

Regardless of ACP cohort, consistent with prior research, traditionally underperforming students 
(Black, Hispanic, economically disadvantaged, at risk, Special Education, and LEP), had lower STAAR 
reading and math scores than their counterparts. This finding may reflect a need for ETF program 
administrators to strengthen their focus of professional development activities on best practices that 
address students’ diverse learning styles and culture, and that are designed to improve performance for 
all students (e.g., Universal Design of Learning). Program strategies could build on the content 
knowledge and pedagogy of teachers at each grade level to strengthen alignment with the TEKS. 
Additional considerations could be made regarding implementing more stringent teacher selection and 
placement practices that take into account the teachers’ preparation and experience, as well as the 
backgrounds and academic needs of targeted students. 
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Appendix A 
ACP Program 


Houston Independent School District ETF (Group 1) Education Service Center (Group 3) 
University/Community College (Group 2) ACP ESC Region IV 

ACP A&M Baylor ACP Harris County Department of Education 

ACP -ACT Houston ACP Region 1 

ACP Cy-Fair College ACP Region 12 

ACP Houston Baptist University ACP Region 20 

ACP Houston Community College Private Entity (Group 4) 

ACP Houston of St. Thomas ACP ATC-East Houston 

ACP Kingwood College ACP Career in Texas 

ACP Lamar University ACP | TEACHTEXAS 

ACP LeTourneau University ACP Texas Teachers 

ACP Prairie View A&M U ACP WebCentric 

ACP Rice University ACP Collin County 

ACP San Jacinto College North ACP Cycle 22C 

ACP Texas Lutheran University ACP Teacher Builder 

ACP U of H - Victoria ACP Texas Alternative Cert 

ACP U of H Downtown Education Career Alternatives 


ACP U of Texas - El Paso 

ACP University of Houston 
ACP University of Phoenix 
ACP Texas Southern University 
Sam Houston PB Intern IN 5 
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Effective Teacher Fellowship (ETF) 
Program Overview 


Application and Selection: 


e =Applicants will complete an online application that includes personal and professional information 
as well as a written assessment and a screening questionnaire. A short math assessment will be 
completed for those candidates seeking any generalists or special education certifications. 

e Applicants must upload all college level transcripts with your online application. Applicants must 
also upload passed THEA scores if the undergraduate overall GPA is below a 3.0, but above a 
2.75. (Note: Once accepted, applicants will need to submit official transcripts of all college level 
course work, including those showing a conferred undergraduate degree.) 

e =Applicants will participate in a structured interview to evaluate candidate mindset. 

e Applicants must pass a complete district screening (including an initial Human Resources 
application screening, a transcript evaluation, and formal background check) to be officially 
accepted into our program. 


Pre-Service Training: 


e Applicants must complete 30 hours of field experience on an HISD campus over the course of the 
district's summer school session (June 2014). 


e =Applicants will be required to attend a number of professional learning courses as scheduled by 
the Effective Teacher Fellowship. 


Internship: 


e Applicants will complete a year-long internship as full-time teachers on campuses across the 
district. 


e Applicants must accumulate 300 hours of professional development throughout their internship. 
This includes attending in-person professional learning courses, completing online modules, and 
participating in educational book studies. 


Certification: 


e Applicants must complete all program requirements as determined by the Effective Teacher 
Fellowship—including a satisfactory year of teaching in an HISD classroom. 
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APPENDIX C 


Profile of ACP Study Sample 


Table 2a. Profile of ETF Student Group, 2012-2013 and 2013-2014 ACP Cohorts 


Subgroup Academic Year 
2012-2013 


2013-2014 


Gender 
Female 2,645 48.8 3,118 49.5 
Male 2,773 51.2 3,187 50.5 
Ethnicity 
American Indian 8 Pak 20 3 
Asian 114 2.1 163 2.6 
African-American 871 16.1 1,776 28.2 
Hispanic 4,231 78.1 4,060 64.4 
White 167 3.1 255 4.0 
Two or More 27 5 31 5 
Economically Disadvantaged 4,860 89.7 5,513 87.4 
At Risk 2,689 49.6 4,408 69.9 
Special Education 336 6.2 494 7.8 
Limited English Proficiency 2,368 43.7 2,196 34.8 


Gifted & Talented (G/T) 


Table 2b. Profile of University/Community College Student Group, 2012-2013 and 2013-2014 ACP Cohorts 


Subgroup Academic Year 
2012-2013 2013-2014 
N % N % 

University/Community College 2,098 100.0 2,105 100.0 
Gender 

Female 1,091 52.0 1,035 49.2 

Male 1,007 48.0 1,070 50.8 
Ethnicity 

American Indian 4 2 Hl <1 

Asian 74 3.5 49 2.3 

African-American 764 36.4 617 29.3 

Hispanic 1,041 49.6 1,232 58.5 

White 196 9.3 197 9.4 

Two or More 19 9 9 4 
Economically Disadvantaged 1,626 77.5 1,596 75.8 
At Risk 887 42.3 1,267 60.2 
Special Education 142 6.8 183 8.7 
Limited English Proficiency 554 26.4 517 24.6 
Gifted & Talented (G/T) 392 18.7 395 18.8 
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APPENDIX C cont’d 


Profile of Study Sample 


Table 2c. Profile of Education Service Center Student Group, 2012-2013 and 2013-2014 ACP Cohorts 


Subgroup Academic Year 


2012-2013 2013-2014 


Education Service Center 


Gender 
Female 1,675 49.8 1,106 48.0 
Male 1,689 50.2 1,198 52.0 
Ethnicity 
American Indian 7 2 7 3 
Asian 75 2.2 55 2.4 
African-American 662 19.7 477 20.7 
Hispanic 2,344 69.7 1,640 71.2 
White 255 7.6 115 5.0 
Two or More 21 6 10 4 
Economically Disadvantaged 2,658 79.0 1,965 85.3 
At Risk 1,760 52.3 1,532 66.5 
Special Education 285 8.5 160 6.9 
Limited English Proficiency 1,054 31.3 915 39.7 


Gifted & Talented (G/T) 


Table 2d. Profile of Private Entity Student Group, 2012-2013 and 2013-2014 ACP Cohorts 


Subgroup Academic Year 


2012-2013 2013-2014 
N % 

Private Entity 
Gender 

Female 9,916 48.5 11,490 48.9 

Male 10,542 51.5 12,027 51.1 
Ethnicity 

American Indian 31 2 55 i2 

Asian 513 2.5 638 2.7 

African-American 6,150 30.1 7,532 32.0 

Hispanic 12,369 60.5 13,764 58.5 

White 1,242 6.1 1,385 5.9 

Two or More 153 7 143 6 
Economically Disadvantaged 16,790 82.1 19,413 82.5 
At Risk 10,144 49.6 15,147 64.4 
Special Education 1,879 9.2 2,066 8.8 
Limited English Proficiency 6,218 30.4 6,284 26.7 
Gifted & Talented (G/T) 3,258 15.9 3,312 14.1 
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2013 STAAR English Reading, 2012-2013 ACP Cohorts 


2012-2013 ACP N] % N] % Mean of 
Cohort N_ Satisfactory Satisfactory Advanced Advanced Scale Score 
ETF 3 180 113 62.8 17 9.4 1362.52 
4 415 206 49.6 30 7.2 1434.88 
5 531 329 62.0 53 10.0 1495.22 
6 1158 768 66.3 177 15.3 1569.90 
7 991 730 73.7 123 12.4 1620.96 
8 752 562 74.7 75 10.0 1633.69 
University/Community 3 81 59 72.8 14 17.3 1414.41 
College 
4 137 96 70.1 26 19.0 1500.46 
5 279 193 69.2 31 11.1 1515.85 
6 283 182 64.3 52 18.4 1569.64 
7 250 182 72.8 25 10.0 1606.19 
8 138 91 65.9 16 11.6 1616.33 
ESC 3 153 104 68.0 13 8.5 1369.97 
4 303 175 57.8 50 16.5 1459.56 
5 235 185 78.7 79 33.6 1576.97 
6 235 164 69.8 45 19.1 1586.26 
7 499 343 68.7 33 6.6 1602.73 
8 325 230 70.8 48 14.8 1644.74 
Private 3 949 730 76.9 201 21.2 1424.30 
4 1594 907 56.9 223 14.0 1463.50 
5 1596 1060 66.4 203 12.7 1512.69 
6 2127 1263 59.4 351 16.5 1558.70 
7 2837 1967 69.3 344 12.1 1611.47 
8 2860 2177 76.1 471 16.5 1652.11 
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APPENDIX D cont’d 


Independent Samples T-test 


2013 STAAR Reading, 2012-2013 ETF vs. Non-ETF Cohorts 
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Mean Mean Std. 

Grade Group N Scale Score | Difference Deviation t p 

3 ETF 180 1362.52 -54.072 124.656 -5.346 -000** 
Non-ETF 1183 1416.59 137.473 

4 ETF 415 1434.88 -30.522 121.091 -4.523 -000*** 
Non-ETF 2034 1465.41 144.074 

5 ETF 531 1495.22 -25.047 111.449 -4.492 -000*** 
Non-ETF 2110 1520.27 127.415 

6 ETF 1158 1569.90 7.583 132.271 1.567 .117 
Non-ETF 2645 1562.32 148.252 

7 ETF 991 1620.96 11.076 113.961 2.694 .007%** 
Non-ETF 3586 1609.88 116.641 

8 ETF 752 1633.69 -16.215 106.341 -3.680 .000*** 
Non-ETF 3323 1649.91 120.546 

Note: Statistical significant at *p < .05, **p < .01, ***p<.001 
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2013 STAAR English Math, 2012-2013 ACP Cohorts 


2012-2013 ACP N % N % Mean of 
(oxo) ato) ay N_ Satisfactory Satisfactory Advanced Advanced _ Scale Score 
ETE 3 172 92 53.5 15 8.7 1407.49 
4 412 231 56.1 38 9.2 1493.25 
5 532 372 69.9 84 15.8 1568.42 
6 1163 816 70.2 184 15.8 1606.24 
7 925 614 66.4 59 6.4 1627.39 
8 623 472 75.8 9 1.4 1648.06 
University/Community 3 83 59 71.1 16 19.3 1489.10 
College 
4 138 77 55.8 13 9.4 1493.64 
5 283 174 61.5 16 5.7 1526.36 
6 277 194 70.0 | 44 15.9 1608.34 
7 235 150 63.8 a a3 1596.43 
8 111 84 75.7 0 0.0 1639.37 
ESC 3 154 75 48.7 11 7.1 1394.37 
4 304 165 54.3 28 9.2 1490.97 
5 240 173 72.1 63 26.3 1595.39 
6 236 162 68.6 41 17.4 1612.12 
7 494 304 61.5 13 2.6 1598.25 
8 255 167 65.5 2 8 1622.42 
Private 3 945 596 63.1 155 16.4 1454.92 
4 1580 881 55.8 185 11.7 1498.69 
5 1598 959 60.0 186 11.6 1534.34 
6 2103 1364 64.9 297 14.1 1589.63 
7 2754 1715 62.3 124 4.5 1607.34 
8 2067 1399 67.7 43 2.1 1633.23 
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Independent Samples T-Test 


2013 STAAR Math, 2012-2013 ETF vs. Non-ETF Cohorts 


Mean Mean 

Grade Group N Scale Score | Difference | Std. Deviation t p 

3 ETF 172 1407.49 -41.941 139.817 -3.616 -000*** 
Non-ETF 1182 1449.43 157.135 

4 ETF 412 1493.25 -3.933 129.991 -551 582 
Non-ETF 2022 1497.18 141.376 

5 ETF 532 1568.42 28.239 133.738 4.286 -000*** 
Non-ETF 2121 1540.18 144.068 

6 ETF 1163 1606.24 12.595 153.965 2.303 .021* 
Non-ETF 2616 1593.64 157.888 

7 ETF 925 1627.39 22.082 134.750 4.542 .000*** 
Non-ETF 3483 1605.31 118.108 

8 ETF 623 1648.06 15.682 91.737 3.759 .000*** 
Non-ETF 2433 1632.38 97.344 

Note: Statistical significant at *p < .05, **p < .01, ***p<.001 
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2014 STAAR English Reading, 2013-2014 ACP Cohorts 


Mean of 
2012-2013 ACP N % N % Scale 
Cohort N_ Satisfactory Satisfactory | Advanced Xo WE Vater] bSYexe) (=) 
ETE 3 544 287 52.8 23 4.2 1345.26 
4 601 397 66.1 44 9.2 1467.98 
5 872 515 59.1 66 7.6 1489.50 
6 762 444 58.3 80 10.5 1534.85 
7 372 225 60.5 45 12.1 1586.33 
8 499 350 70.1 78 15.6 1641.46 
University/Community 3 15 4 26.7 1 6.7 1319.27 
College oe __ 
4 67 36 53.7 4 6.0 1441.81 
5 272 193 71.0 42 15.4 1526.67 
6 288 255 88.5 62 21.5 1633.95 
7 31 23 74.2 5 16.1 1636.74 
8 268 169 63.1 10 3.7 1605.14 
ESC 3 47 18 38.3 1 2.1 1305.43 
4 331 192 58.0 21 6.3 1442.92 
5 171 109 63.7 25 14.6 1527.83 
6 227 143 63.0 15 6.6 1543.81 
7 208 98 47.1 11 5.3 1548.69 
8 406 333 82.0 81 20.0 1670.11 
Private 3 1045 620 59.3 89 8.5 1369.60 
4 1286 706 54.9 117 9.1 1442.89 
5 1573 4956 £608 177 113 1503.22 | 
6 2779 1836 66.1 267 9.6 1552.23 
7 3497 2299 65.7 481 13.8 1602.05 
8 2683 1991 74.2 325 12.1 1642.53 
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APPENDIX E cont’d 


Independent T-test 


2014 STAAR Reading, 2013-2014 ETF vs. Non-ETF Cohorts 


Mean Mean 

Grade Group N Scale Score Difference Std. Deviation t p 

3 ETF 544 1345.26 -20.939 118.191 -3.281 .001** 
Non-ETF 1107 1366.20 129.075 

4 ETF 601 1467.98 25.123 115.309 4.530 .000*** 
Non-ETF 1684 1442.85 120.606 

5 ETF 872 1489.50 -18.976 119.280 -3.799 -000*** 
Non-ETF 2016 1508.47 131.910 

6 ETF 762 1534.85 -23.939 132.994 -4,520 -000*** 
Non-ETF 3294 1558.79 126.235 

7 ETF 372 1586.33 -13.040 133.632 -1.809 O71 
Non-ETF 3736 1599.37 121.714 

8 ETF 499 1641.46 -1.414 121.425 -.245 -807 
Non-ETF 3357 1642.88 113.916 

Note: Statistical significant at *p < .05, **p < .01, ***p<.001 
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APPENDIX E cont’d 


2014 STAAR English Math, 2013-2014 ACP Cohorts 


2013-2014 ACP N % N % Mean of 

foro) are) a N_ Satisfactory Satisfactory Advanced Advanced Scale Score 

1392.98 

4 606 404 66.7 102 16.8 1524.07 

5 870 595 68.4 132 15.2 1558.56 

6 736 445 60.5 91 12.4 1579.15 

7 335 214 63.9 29 8.7 1589.44 

8 414 278 67.1 17 4.1 1638.95 

University/Community 3 15 4 26.7 0 0.0 1281.27 
College 

4 67 36 53.7 5 7.5 1455.37 

5 273 206 75.5 64 23.4 1598.08 

6 287 255 88.9 71 24.7 1670.67 

7 31 24 77.4 5 16.1 1646.77 

8 236 130 55.1 5 2.1 1604.13 

ESC 3 47 18 38.3 2 4.3 1319.17 

4 339 199 58.7 52 15.3 1529.42 

5 170 147 86.5 42 24.7 1632.04 

6 214 144 67.3 30 14.0 1596.79 

7 187 93 49.7 5 2.7 1552.52 

8 241 176 73.0 1 4 1630.20 

Private 3 1043 626 60.0 113 10.8 1420.46 

4 1286 711 55.3 162 12.6 1496.47 

5 1563 1056 67.6 249 15.9 1559.30 

6 2726 1849 67.8 413 15.2 1600.11 

7 3331 2226 66.8 272 8.2 1598.70 

8 2134 1556 72.9 78 3.6 1647.18 
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Independent T-test 


2014 STAAR Math, 2013-2014 ETF vs. Non-ETF Cohorts 


Mean Mean 

Grade Group N Scale Score Difference Std. Deviation t p 

3 ETF 542 1392.98 -21.281 143.106 -2.730 .006** 
Non-ETF 1105 1414.26 159.390 

4 ETF 606 1524.07 22.629 150.394 3.209 .001** 
Non-ETF 1692 1501.45 144.801 

5 ETF 870 1558.56 -12.186 140.692 -2.105 .035* 
Non-ETF 2006 1570.74 146.836 

6 ETF 736 1579.15 -27.013 153.650 -4,345 -000*** 
Non-ETF 3227 1606.17 151.862 

7 ETF 335 1589.44 -7.245 129.687 -1.032 -302 
Non-ETF 3549 1596.69 122.109 

8 ETF 414 1638.95 -2.768 119.486 -.486 -627 
Non-ETF 2611 1641.72 105.650 

Note: Statistical significant at *p < .05, **p < .01, ***p<.001 
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2013 STAAR End-of-Course, 2012-2013 ACP Cohorts 


2013 STAAR EOC Results by Exam for the 2012-2013 ACP Teacher/Student Cohorts 


Mean Std. Percent Percent 
N Minimum Maximum Scale Score Deviation Satisfactory Advanced 

ETF Algebra | 424 2984 6127 3942.68 519.259 78.5 21.2 
ELAI- 426 1227 2638 1909.58 228.653 63.4 3:5 
Reading 
ELAI- 444 1274 2650 1798.02 213.065 38.7 2 
Writing 

University/ Algebra | 248 2846 6127 3938.55 438.871 86.3 15.3 

Community 

College ELA I - 363 1324 3298 2088.68 228.995 88.4 15.7 
Reading 
ELA - 363 1383 3228 1990.76 235.296 73.8 3.0 
Writing 

ESC Algebra | 439 2846 5588 3734.82 466.012 68.1 11.6 
ELA I - 550 1324 3298 1920.93 303.145 55.3 10.2 
Reading 
ELA I- 578 424 3228 1823.58 297.408 39.3 3.3 
Writing 

Private Algebra | 2989 1367 6127 3803.65 475.155 74.0 12.1 
ELA I - 2913 661 3298 1886.11 251.579 53.2 5.3 
Reading 
ELAI- 3037 424 3228 1784.96 250.603 35.2 8 
Writing 


Mean Mean 

EOC Exam | Group N Scale Score Difference Std. Deviation t p 
Algebra 1 ETF 424 3942.68 138.147 519.259 5.233 .000*** 

Non-ETF 3676 3804.53 473.485 
ELA I- ETF 426 1909.58 -753 228.653 .063 .949 
Reading 

Non-ETF 3826 1910.34 264.233 
ELA I- ETF 444 1798.02 -1.338 213.065 -1.036 300 
Writing 

Non-ETF 3978 1809.35 263.253 
Note: Statistical significant at *p < .05, **p < .01, ***p<.001 
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2014 STAAR End-of-Course, 2013-2014 ACP Cohorts 


2014 STAAR EOC Scale Score Results by Exam for the 2013-2014 ACP Teacher/Student Cohorts 


Mean 
Scale Std. Percent Percent 
N Minimum | Maximum Score | Deviation Satisfactory | Advanced 
ETF Algebra | 761 1379 6116 3721.57 436.431 73.6 14.0 
English | 1026 2556 5443 3677.99 403.605 46.5 1.0 
Algebra | 162 2985 4820 3590.43 371.685 79.5 21.7 
culece Community |“English | 253 2871 6055 | 3736.44 | 477.333 49.6 3.2 
Algebra | 457 1379 5577 3810.82 417.926 82.2 11.5 
ESC English | 417 2871 5095 3759.76 395.851 55.3 1.9 
Private Algebra | 2974 1379 6116 3826.92 478.752 80 20.0 
English | 4106 1075 6966 3783.42 466.976 56.2 5.0 


Mean Mean 
EOC Exam | Group N Scale Score Difference Std. Deviation t p 
Algebra 1 ETF 761 3721.57 -92.635 436.431 -5.247 -000*** 
Non-ETF 3593 3814.21 469.536 
English | ETF 1026 3677.99 -100.878 403.605 -7.073 -000*** 
Non-ETF 4776 3778.87 461.834 
Note: Statistical significant at *p < .05, **p < .01, ***p<.001 
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2013 Stanford 10 Reading, 2012-2013 ACP Cohorts 


2012-2013 Reading WCET eS Std. Deviation 
EE 4 — 26.27 19.972 
2 69 31.23 20.587 
3 186 37.87 19.932 
4 434 36.84 16.596 
5 550 38.59 17.883 
6 1191 43.44 20.879 
7 1014 44.11 18.961 
8 758 40.44 15.778 
University/Community 1 100 41.11 18.363 
College 
2 102 42.77 22.052 
3 87 48.30 19.321 
4 144 48.34 21.430 
5 294 40.38 16.814 
6 298 41.94 22.813 
7 253 42.53 19.135 
8 139 36.02 19.077 
ESC 1 138 40.27 23.445 
2 146 41.19 21.283 
3 154 39.81 18.442 
4 302 41.23 20.844 
5 248 51.70 25.732 
6 247 46.60 22.722 
7 520 38.97 18.319 
8 335 41.48 19.948 
Private 1 1041 41.64 22.405 
2 781 41.54 20.664 
3 986 49.34 22.223 
4 1651 42.29 20.987 
5 1695 40.33 20.325 
6 2261 40.99 23.909 
7 2977 41.44 21.279 
8 2979 43.73 18.850 
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APPENDIX H cont’d 


2013 Stanford 10 Math, 2012-2013 ACP Cohorts 


2012-2013 Math Mean NCE BS} Co Mg BY =\VE-Vice) a) 
ETF 1 153 38.98 18.207 
2 70 39.75 19.092 
3 186 47.76 21.515 
4 434 49.28 18.582 
5 550 51.76 19.476 
6 1187 51.89 19.950 
7 1017 57.28 22.484 
8 757 51.85 16.853 
University/Community 1 101 48.33 19.356 
College 
2 102 47.84 22.782 
3 88 59.43 22.639 
4 144 54.11 21.026 
5 295 47.50 17.910 
6 298 52.68 19.454 
7 251 51.09 21.385 
8 139 51.22 18.833 
ESC 1 139 40.90 20.782 
2 147 41.91 19.030 
3 155 49.67 20.787 
4 302 50.66 21.119 
5 248 56.83 23.504 
6 247 51.95 20.014 
7 520 49.53 19.719 
8 334 48.42 19.892 
Private 1 1046 44.62 20.625 
2 780 44.99 19.108 
3 985 55.43 23.584 
4 1651 50.84 21.086 
5 1698 49.03 20.576 
6 2257 48.73 21.444 
7 2974 51.17 21.629 
8 2971 53.54 19.420 
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2014 Stanford 10 Reading, 2013-2014 ACP Cohorts 


2013-2014 Reading WET Coss Std. Deviation 
ETF 1 88 33.97 20.517 
2 378 36.32 18.655 
3 562 35.88 20.295 
4 630 40.86 17.011 
5 924 36.55 18.501 
6 821 37.02 23.188 
7 396 37.04 21.221 
8 537 41.40 20.334 
University/Community 1 55 29.59 19.718 
College 
2 21 31.73 21.353 
3 15 33.13 20.260 
4 68 39.21 18.103 
5 288 42.06 20.367 
6 290 54.00 20.344 
7 35 40.88 23.729 
8 273 38.69 16.083 
ESC 1 36 41.20 21.428 
2 43 51.91 22.009 
3 52 34.04 19.425 
4 347 38.87 16.602 
5 177 44.84 23.149 
6 234 38.28 20.877 
7 223 30.19 19.668 
8 413 45.81 18.145 
Private 1 1006 38.69 23.003 
2 792 35.30 20.703 
3 1090 40.33 21.590 
4 1327 39.63 19.244 
5 1654 40.13 20.958 
6 2956 40.24 22.853 
7 3637 40.26 21.313 
8 2802 42.70 18.321 
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APPENDIX | cont’d 


2014 Stanford 10 Math, 2013-2014 ACP Cohorts 


2013-2014 Math Mean NCE 5S) (om DY=NVr- Lice) a) 


ETF 1 88 45.47 20.318 
2 387 41.26 21.279 
3 566 45.75 22.160 
4 630 51.20 18.474 
5 926 46.48 20.163 
6 823 45.70 20.651 
7 396 46.92 22.585 
8 531 49.68 19.950 
University/Community 1 54 37.62 18.279 
College jee 
2 21 40.26 24.229 
3 15 33.76 17.863 
4 65 44.00 17.937 
5 288 51.21 21.622 
6 290 60.78 18.293 
7 35 51.41 21.759 
8 262 44.75 16.723 
ESC 1 36 44.81 18.038 
2 43 52.85 23.235 
3 52 40.09 18.376 
4 347 51.69 17.621 
5 177 58.39 20.642 
6 234 50.20 19.287 
7 222 43.43 19.959 
8 414 50.91 17.417 
Private 1 1010 44.75 22.083 
2 797 39.68 20.830 
3 1095 48.42 23.382 
4 1325 48.19 20.614 
5 1651 46.99 21.418 
6 2948 49.57 20.730 
7 3615 49.97 21.940 
8 2797 52.13 19.157 
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APPENDIX J 


Value-added Composite Gain Index 


-2 or below (“Well Below”) 
Students in the district made substantially less progress than the Standard for Academic Growth 
(growth measure is below the Standard for Academic Growth by more than 2 standard errors). 


-2 to -1 (“Below”) 

Students in the district made less progress than the Standard for Academic Growth (growth 
measure is more than 1 standard error below the Standard for Academic Growth but by 2 
standard errors or less). 


-1 to 1 (“No Detectable Difference”) 

Students in the district made progress similar to the Standard for Academic Growth (growth 
measure is at most 1 standard error below the Standard for Academic Growth but less than 1 
standard error above it). 


1 to 2 (“Above”) 

Students in the district made more progress than the Standard for Academic Growth (growth 
measure is above the Standard for Academic Growth by at least 1 standard error but less than 2 
standard errors above it). 


2 or above (“Well Above”) 

Students in the district made substantially more progress than the Standard for Academic 
Growth (growth measure is above the Standard for Academic Growth by 2 standard errors or 
more). 
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